1. Field
Certain aspects of the present disclosure generally relate to neural system engineering and, more particularly, to a method and apparatus for utilizing a memory in probabilistic manner to store information about synaptic weights of a neural network.
2. Background
Neural networks can have hundreds of thousands of synapses, wherein weights of the synapses can be learned during network training During the course of training, the weights are typically initialized with random values and changed in small increments. Often, the synaptic weights are stored in a multi-bit or multi-level memory. However, in many cases, the weights may settle to one of two values (bimodal distribution of weights). Therefore, using the multi-bit memory to store final binary weights can be waste of storage resources. Moreover, addressing the multi-bit synaptic memory during network training or operation can be a bottleneck of speed.